Papers by Makesh Narsimhan Sreedhar
Guardrails and Security for LLMs: Safe, Secure and Controllable Steering of LLM Applications (2025.acl-tutorials)
Copied to clipboard
Traian Rebedea, Leon Derczynski, Shaona Ghosh, Makesh Narsimhan Sreedhar, Faeze Brahman, Liwei Jiang, Bo Li, Yulia Tsvetkov, Christopher Parisien, Yejin Choi
| Challenge: | Pretrained generative models provide novel ways for users to interact with computers. |
| Approach: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs along with evaluation methodologies and a detailed security assessment protocol. |
| Outcome: | This tutorial provides an overview of key guardrail mechanisms developed for LLMs, along with evaluation methodologies and a detailed security assessment protocol. |
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails (2023.emnlp-demo)
Copied to clipboard
| Challenge: | NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. |
| Approach: | They propose to add programmable guardrails to LLMs that are user-defined, independent of the underlying LLM, and interpretable. |
| Outcome: | The proposed approach can be used with several LLM providers to develop controllable and safe LLM applications using programmable rails. |
Unsupervised Extraction of Dialogue Policies from Conversations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used to extract dialogue policies from conversational data. |
| Approach: | They propose a method for extracting dialogue policies from conversational data using canonical forms and graph traversal algorithms. |
| Outcome: | The proposed method gives conversation designers greater control and improves the process of developing dialogue policies. |
Single Sequence Prediction over Reasoning Graphs for Multi-hop QA (2023.acl-long)
Copied to clipboard
| Challenge: | Recent generative approaches for multi-hop question answering (QA) use fusion-in-decoder to generate a single sequence output . but, they often have difficulty accurately identifying passages corresponding to key entities in the context . |
| Approach: | They propose a single-sequence prediction method that integrates a graph structure linking key entities in each context passage to relevant subsequent passages for each question. |
| Outcome: | The proposed method improves answer exact-match/F1 scores and faithfulness of grounding on the hotpotQA dataset and achieves state-of-the-art numbers on the Musique dataset. |
CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in instruction-tuning datasets focus on specific tasks like mathematical or logical reasoning. |
| Approach: | They propose to use synthetic dialogues to help language models remain focused on the subject at hand during task-oriented interactions. |
| Outcome: | The proposed dataset improves language models' ability to maintain topical coherence compared to general-purpose instruction-tuned LLMs like gpt-4-turbo and Mixtral-Instruct. |
Learning Improvised Chatbots from Adversarial Modifications of Natural Language Feedback (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, user feedback contains extraneous sequences hindering their usefulness as a training sample. |
| Approach: | They propose a generative adversarial model that converts noisy feedback into a plausible natural response in a conversation and fools the discriminator which distinguishes feedback from natural responses. |
| Outcome: | The proposed model improves the original chatbot performance from 69.94%to 75.96% in ranking correct responses on the PERSONACHATdataset. |
AEGIS2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails (2025.naacl-long)
Copied to clipboard
Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea, Jibin Rajan Varghese, Christopher Parisien
| Challenge: | Existing safety-related content safety models are not well-suited for commercial use. |
| Approach: | They propose a taxonomy that can be used to categorize safety risks . it combines human annotations with a multi-LLM "jury" system to assess safety . they plan to open-source Aegis2.0 data and models to aid in safety guardrailing . |
| Outcome: | The proposed taxonomy can be used to assess the safety of human-LLM interactions . it can be trained on large, non-commercial datasets and is open-source . |
Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research shows that reasoning-based language models offer significant benefits for LLM safety and guardrail applications. |
| Approach: | They conduct an analysis of reasoning-based guardrail models for content moderation . they find reasoning models exhibit strong sample efficiency and inference efficiency . |
| Outcome: | The reasoning-based guardrail models show strong performance across domains . the models achieve competitive performance with significantly fewer training examples . |
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM (2024.naacl-long)
Copied to clipboard
Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev
| Challenge: | Existing helpfulness preference datasets do not specify what makes some responses more helpful and others less helpful. |
| Approach: | They use a dataset that has annotated for correctness, coherence, complexity, and verbosity. |
| Outcome: | The dataset has annotations for correctness, coherence, complexity, and verbosity in addition to overall helpfulness of responses. |
Local Byte Fusion for Neural Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing NLP models rely on a pre-built subword tokenizer to tokenize a sentence . this can be rigid and subwords from low-resource languages are under-represented . |
| Approach: | They propose a method for byte-based machine translation that aggregates local semantic information. |
| Outcome: | The proposed method improves on multilingual translation and cross-lingual transfer . it is parameter-efficient and performs competitively to subword models, it is shown . |